Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: Add cargo bench for shuffle writer #438

Merged
merged 3 commits into from
May 16, 2024

Conversation

andygrove
Copy link
Member

@andygrove andygrove commented May 16, 2024

Which issue does this PR close?

N/A

Rationale for this change

I would like to profile the shuffle writer code and understand if there are any opportunities for optimizing performance.

What changes are included in this PR?

  • Add a new benchmark for shuffle writer.
  • Add new section to contributors guide
Screenshot 2024-05-16 at 4 07 34 PM

How are these changes tested?

@codecov-commenter
Copy link

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 34.17%. Comparing base (fcf7d5b) to head (f8b3203).
Report is 3 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff            @@
##               main     #438   +/-   ##
=========================================
  Coverage     34.17%   34.17%           
  Complexity      850      850           
=========================================
  Files           116      116           
  Lines         38547    38552    +5     
  Branches       8523     8524    +1     
=========================================
+ Hits          13172    13175    +3     
- Misses        22609    22610    +1     
- Partials       2766     2767    +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@andygrove andygrove marked this pull request as ready for review May 16, 2024 22:08
@andygrove andygrove requested a review from viirya May 16, 2024 22:08
Copy link
Member

@viirya viirya left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me. This will be useful for contributors. Thanks @andygrove

@andygrove
Copy link
Member Author

Thanks for the review @viirya

@andygrove andygrove merged commit f8fec7f into apache:main May 16, 2024
41 checks passed
@andygrove andygrove deleted the shuffle-write-bench branch May 16, 2024 23:56
himadripal pushed a commit to himadripal/datafusion-comet that referenced this pull request Sep 7, 2024
* Add cargo bench for shuffle writer

* add profiling guide

* add sample output

(cherry picked from commit f8fec7f)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants